MMH: Maximum Margin Supervised Harmoniums

Exponential family Harmoniums (EFH) are undirected topic models that enjoy nice properties such as fast inference compared to directed topic models. Supervised EFHs can utilize documents’ side information for discovering predictive latent topic representations. However, existing likelihood based estimation does not yield conclusive results. This paper presents a max-margin approach to learning supervised EFHs for joint latent topic discovery and classification. The learning problem is efficiently solved with coordinate descent. We demonstrate the advantages of the max-margin approach on video data classification and retrieval.